Speaker and Noise Factorisation for Robust Speech Recognition

Speech recognition systems need to operate in a wide range of conditions. Thus they should be robust to extrinsic variability caused by various acoustic factors, for example speaker differences, transmission channel and background noise. For many scenarios, multiple factors simultaneously impact the underlying “clean” speech signal. This paper examines techniques to handle both speaker and background noise differences. An acoustic factorisation approach is adopted. Here separate transforms are assigned to represent the speaker (maximum likelihood linear regression (MLLR)), and noise and channel (model-based vector Taylor series (VTS)) factors. This is a highly flexible framework compared to the standard approaches of modelling the combined impact of both speaker and noise factors. For example factorisation allows the speaker characteristics obtained in one noise condition to be applied to a different environment. To obtain this factorisation modified versions of MLLR and VTS training and application are derived. The proposed scheme is evaluated for both adaptation and factorisation on the AURORA4 data.